Contributions to metric-topological localization and mapping in mobile robotics

This thesis addresses the problem of localization and mapping in mobile robotics. The ability of a robot to build a map of an unknown environment from sensory information is required to perform self-localization and autonomous navigation, as a necessary condition to carry out more complex tasks. This problem has been widely investigated in the last decades, but the solutions presented have still important limitations, mainly to cope with large scale and dynamic environments, and to work in a wider range of conditions and scenarios. In this context, this thesis takes a step forward towards highly efficient localization and mapping.A first contribution of this work is a new mapping strategy that presents two key features: the lightweight representation of world metric information, and the organization of this metric map into a topological structure that allows efficient localization and map optimization. Regarding the first issue, a map is proposed based on planar patches which are extracted from range or RGB-D images. This plane-based map (PbMap) is particularly well suited for indoor scenarios, and has the advantage of being a very compact and still a descriptive representation which is useful to perform real-time place recognition and loop closure. These operations are based on matching planar features taking into account their geometric relationships. On the other hand, the abstraction of metric information is necessary to deal with large scale SLAM and with navigation in complex environments. For that, we propose to structure the map in a metric-topological structure which is dynamically organized upon the sensor observations.  Also, a simultaneous localization and mapping (SLAM) system employing an omnidirectional RGB-D device which combines several structured-light sensors (Asus Xtion Pro Live) is presented. This device allows the quick construction of rich models of the environment at a relative low cost in comparison with previous alternatives. Our SLAM approach is based on a hierarchical structure of keyframes with a low level layer of metric information and several topological layers intended for large scale SLAM and navigation. This SLAM solution, which makes use of the metric-topological representation mentioned above, works at video frame rate obtaining highly consistent maps. Future research is expected on metric-topological-semantic mapping from the new sensor and the SLAM system presented here. Finally, an extrinsic calibration technique is proposed to obtain the relative poses of a combination of 3D range sensors, like those employed in the omnidirectional RGB-D device mentioned above. The calibration is computed from the observation of planar surfaces of a structured environment in a fast, easy and robust way, presenting qualitative and quantitative advantages with respect to previous approaches. This technique is extended to calibrate any combination of range sensors, including 2D and 3D range sensors, in any configuration. The calibration of such sets of sensors is interesting not only for mobile robots, but also for autonomous cars

On the Production of Semantic and Textured 3D Meshes of Large scale Urban Environments from Mobile Mapping Images and LiDAR scans

International audienceDans cet article nous présentons un cadre entièrement au-tomatique pour la reconstruction d'un maillage, sa textu-ration et sa sémantisation à large échelle à partir de scans LiDAR et d'images orientées de scènes urbaines collectés par une plateforme de cartographie mobile terrestre. Tout d'abord, les points et les images georéferencés sont dé-coupés temporellement pour assurer une cohèrence entre la geométrie (les points) et la photométrie (les images). Ensuite, une reconstruction de surface 3D simple et ra-pide basée sur la topologie d'acquisition du capteur est effectuée sur chaque segment après un rééchantillonnage du nuage de points obtenu à partir des balayages LiDAR. L'algorithme de [31] est par la suite adapté pour texturer la surface reconstruite avec les images acquises simultané-ment assurant une texture de haute qualité et un ajustement photométrique global. Enfin, en se basant sur le schéma de texturation, une sémantisation par texel est appliquée sur le modèle final. Mots Clef scène urbaine, cartographie mobile, LiDAR, reconstruction de surface, texturation, sémantisation, apprentissage pro-fond. Abstract In this paper we present a fully automatic framework for the reconstruction of a 3D mesh, its texture mapping and its semantization using oriented images and LiDAR scans acquired in a large urban area by a terrestrial Mobile Mapping System (MMS). First, the acquired points and images are sliced into temporal chunks ensuring a reasonable size and time consistency between geometry (points) and pho-tometry (images). Then, a simple and fast 3D surface reconstruction relying on the sensor space topology is performed on each chunk after an isotropic sampling of the point cloud obtained from the raw LiDAR scans. The method of [31] is subsequently adapted to texture the reconstructed surface with the images acquired simultaneously, ensuring a high quality texture and global color adjustment. Finally, based on the texturing scheme a per-texel semantization is conducted on the final model

A dense map building approach from spherical RGBD images

International audienceVisual mapping is a required capability for practical autonomous mobile robots where there exists a grow- ing industry with applications ranging from the service to industrial sectors. Prior to map building, Visual Odometry(VO) is an essential step required in the process of pose graph construction. In this work, we first propose to tackle the pose estimation problem by using both photometric and geometric information in a direct RGBD image registration method. Secondly, the mapping problem is tackled with a pose graph representation, whereby, given a database of augmented visual spheres, a travelled trajectory with redundant information is pruned out to a skeletal pose graph. Both methods are evaluated with data acquired with a recently proposed omnidirectional RGBD sensor for indoor environments

Extrinsic calibration of a set of range cameras in 5 seconds without pattern

International audienceThe integration of several range cameras in a mobile platform is useful for applications in mobile robotics and autonomous vehicles that require a large field of view. This situation is increasingly interesting with the advent of low cost range cameras like those developed by Primesense. Calibrating such combination of sensors for any geometric configuration is a problem that has been recently solved through visual odometry (VO) and SLAM. However, this kind of solution is laborious to apply, requiring robust SLAM or VO in controlled environments. In this paper we propose a new uncomplicated technique for extrinsic calibration of range cameras that relies on finding and matching planes. The method that we present serves to calibrate two or more range cameras in an arbitrary configuration, requiring only to observe one plane from differ- ent viewpoints. The conditions to solve the problem are studied, and several practical examples are presented covering different geometric configurations, including an omnidirectional RGB- D sensor composed of 8 range cameras. The quality of this calibration is evaluated with several experiments that demon- strate an improvement of accuracy over design parameters, while providing a versatile solution that is extremely fast and easy to apply

Reducing the environmental impact of surgery on a global scale: systematic review and co-prioritization with healthcare workers in 132 countries

Abstract Background Healthcare cannot achieve net-zero carbon without addressing operating theatres. The aim of this study was to prioritize feasible interventions to reduce the environmental impact of operating theatres. Methods This study adopted a four-phase Delphi consensus co-prioritization methodology. In phase 1, a systematic review of published interventions and global consultation of perioperative healthcare professionals were used to longlist interventions. In phase 2, iterative thematic analysis consolidated comparable interventions into a shortlist. In phase 3, the shortlist was co-prioritized based on patient and clinician views on acceptability, feasibility, and safety. In phase 4, ranked lists of interventions were presented by their relevance to high-income countries and low–middle-income countries. Results In phase 1, 43 interventions were identified, which had low uptake in practice according to 3042 professionals globally. In phase 2, a shortlist of 15 intervention domains was generated. In phase 3, interventions were deemed acceptable for more than 90 per cent of patients except for reducing general anaesthesia (84 per cent) and re-sterilization of ‘single-use’ consumables (86 per cent). In phase 4, the top three shortlisted interventions for high-income countries were: introducing recycling; reducing use of anaesthetic gases; and appropriate clinical waste processing. In phase 4, the top three shortlisted interventions for low–middle-income countries were: introducing reusable surgical devices; reducing use of consumables; and reducing the use of general anaesthesia. Conclusion This is a step toward environmentally sustainable operating environments with actionable interventions applicable to both high– and low–middle–income countries

Contributions to metric-topological localization and mapping in mobile robotics

Advisor: Javier Gonzalez-Jiménez. PhD thesis defended 27th October 2014, Universidad de MalagaThis thesis addresses the problem of localization and mapping in mobile robotics. The ability of a robot to build a map of an unknown environment from sensory information is required to perform self-localization and autonomous navigation, as a necessary condition to carry out more complex tasks. This problem has been widely investigated in the last decades, but the solutions presented have still important limitations, mainly to cope with large scale and dynamic environments, and to work in a wider range of conditions and scenarios. In this context, this thesis takes a step forward towards highly efficient localization and mapping.A first contribution of this work is a new mapping strategy that presents two key features: the lightweight representation of world metric information, and the organization of this metric map into a topological structure that allows efficient localization and map optimization. Regarding the first issue, a map is proposed based on planar patches which are extracted from range or RGB-D images. This plane-based map (PbMap) is particularly well suited for indoor scenarios, and has the advantage of being a very compact and still a descriptive representation which is useful to perform real-time place recognition and loop closure. These operations are based on matching planar features taking into account their geometric relationships. On the other hand, the abstraction of metric information is necessary to deal with large scale SLAM and with navigation in complex environments. For that, we propose to structure the map in a metric-topological structure which is dynamically organized upon the sensor observations. Also, a simultaneous localization and mapping (SLAM) system employing an omnidirectional RGB-D device which combines several structured-light sensors (Asus Xtion Pro Live) is presented. This device allows the quick construction of rich models of the environment at a relative low cost in comparison with previous alternatives. Our SLAM approach is based on a hierarchical structure of keyframes with a low level layer of metric information and several topological layers intended for large scale SLAM and navigation. This SLAM solution, which makes use of the metric-topological representation mentioned above, works at video frame rate obtaining highly consistent maps. Future research is expected on metric-topological-semantic mapping from the new sensor and the SLAM system presented here. Finally, an extrinsic calibration technique is proposed to obtain the relative poses of a combination of 3D range sensors, like those employed in the omnidirectional RGB-D device mentioned above. The calibration is computed from the observation of planar surfaces of a structured environment in a fast, easy and robust way, presenting qualitative and quantitative advantages with respect to previous approaches. This technique is extended to calibrate any combination of range sensors, including 2D and 3D range sensors, in any configuration. The calibration of such sets of sensors is interesting not only for mobile robots, but also for autonomous cars

Adaptive Direct RGB-D Registration and Mapping for Large Motions

International audienceDense direct RGB-D registration methods are widely used in tasks ranging from localization and tracking to 3D scene reconstruction. This work addresses a peculiar aspect which drastically limits the applicability of direct registration, namely the weakness of the convergence domain. First, we propose an activation function based on the conditioning of the RGB and ICP point-to-plane error terms. This function strengthens the geometric error influence in the first coarse iterations, while the intensity data term dominates in the finer increments. The information gathered from the geometric and photometric cost functions is not only considered for improving the system observability, but for exploiting the different convergence properties and convexity of each data term. Next, we develop a set of strategies as a flexible regularization and a pixel saliency selection to further improve the quality and robustness of this approach. The methodology is formulated for a generic warping model and results are given using perspective and spherical sensor models. Finally, our method is validated in different RGB-D spherical datasets, including both indoor and outdoor real sequences and using the KITTI VO/SLAM benchmark dataset. We show that the different proposed techniques (weighted activation function, regularization, saliency pixel selection), lead to faster convergence and larger convergence domains, which are the main limitations to the use of direct methods

An Efficient Rotation and Translation Decoupled Initialization from Large Field of View Depth Images

International audienceImage and point cloud registration methods compute the relative pose between two images. Commonly used registration algorithms are iterative and rely on the assumption that the motion between the images is small. In this work, we propose a fast pose estimation technique to compute a rough estimate of large motions between depth images, which can be used as initialization to dense registration methods. The main idea is to explore the properties given by planar surfaces with co-visibility and their normals from two distinct viewpoints. We present, in two decoupled stages, the rotation and then the translation estimation, both based on the normal vectors orientation and on the depth. These two stages are efficiently computed by using low resolution depth images and without any feature extraction/matching. We also analyze the limitations and observabilty of this approach, and its relationship to ICP point-to-plane. Notably, if the rotation is observable, at least five degrees of freedom can be estimated in the worst case. To demonstrate the effectiveness of the method, we evaluate the initialization technique in a set of challenging scenarios, comprising simulated spherical images from the Sponza Atrium model benchmark and real spherical indoor sequences

Dense Accurate Urban Mapping from Spherical RGB-D Images

International audience— This paper presents a methodology to combine information from a sequence of RGB-D spherical views acquired by a home-made multi-stereo device in order to improve the computed depth images both in terms of accuracy and completeness. This methodology is embedded in a larger visual mapping framework aiming to produce accurate and dense topometric urban maps. Our method is based on two main filtering stages. Firstly, we perform a segmentation process considering both geometric and photometric image constraints, followed by a regularization step (spatial-integration). We then proceed to a fusion stage where the geometric information is further refined by considering the depth images of nearby frames (temporal integration). This methodology can be applied to other projective models, such as perspective stereo images. Our approach is evaluated within the frameworks of image registration, localization and mapping, demonstrating higher accuracy and larger convergence domains over different datasets

A new metric for evaluating semantic segmentation: leveraging global and contour accuracy

International audienceSemantic segmentation of images is an important problem for mobile robotics and autonomous driving because it offers basic information which can be used for complex reasoning and safe navigation. Different solutions have been proposed for this problem along the last two decades, and a relevant increment on accuracy has been achieved recently with the application of deep neural networks for image segmentation. One of the main issues when comparing different neural networks architectures is how to select an appropriate metric to evaluate their accuracy. Furthermore, commonly employed evaluation metrics can display divergent outcomes, and thus it is not clear how to rank different image segmentation solutions. This paper proposes a new metric which accounts for both global and contour accuracy in a simple formulation to overcome the weaknesses of previous metrics. We show with several examples the suitability of our approach and present a comparative analysis of several commonly used metrics for semantic segmentation together with a statistical analysis of their correlation. Several network segmentation models are used for validation with virtual and real benchmark image sequences, showing that our metric captures information of the most commonly used metrics in a single scalar value